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"Determiriing the Readability of Instructional Material" 
SIXTH WORLD CONGRESS ON P^ADTNG, SINGAPORE, August, 1976 
Session? Readability, August 18, 1976 

Insti^uctional materials comprising commercially produced 
texts and supplementary work-study books and including teach- 
er prepared booklets and study sheets are a vital aspect of 
all educatioiial endeavors. They are effective means for in- 
troducing or overviewing a topic, for providing a context for 
introducing new terms and concepts, for gaining specific in- 



formation and common experiences, and for summarizing and 
concluding the study of a topic . Whether texts should be 
used in a particular educational situation is a decision that 
must be made by the local educational unit and staff < The 
aim of this paper is to present some ideas that have implica-^ 
tions for the effective selection of instructional materials 
whenever the decision is made to use them. 

One concern that seems to constantly appear in the lit= 
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erature on the use of reading materials is about their reada- 
bility , No one seems to disagree that instructional materials 
should be demonstrably "readable" by the pupils. V/hat seems 
to be a source of disagreement, and sometimes a source of 
confusion » i s the means aval lable for determining v/hether 
or not a set of materials is readable. The disagreement and 
confusion seems to result from whether one is attempting to 
predict or to measure the degree of readability of any text* 

DEFINING READABILITY 

There is substantial agreement among authors, on the sub- 
ject of readability as to what may affect the understanding 
of a particular piece of 'written material. Fev/ define It 
simply as factors or qualities v/ithln the material Itself. 
.Most Include in the definitions not only factors that re- 
flect the lariguage characteristics of the written message 
but also those that relfect characteristics of the reader 
and those that take into account the difficulty of the ideas 
within the subject matter* Some also include characteristics 
of the affective nature of the reading act. For axample, 
readability could be considered as (7) 

the emse with which llngiilstlc material in writ-' 
ten form with (given) cognitive or emotional 
characteristics of content or style and pre^ 
' sented in a (given) manner with or without sup- 
porting context is decoded-»understood , learned, 
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Hittleman ^3 
remembered«^=by members of a (given) population. 

In a similar vein, readability is considered as an es- 
timate of reading difficulty at certain times for certain 
purposes, and (2^) 

the reader's level of performance 

is a function of 
the reader- s level of competence 

Interacting with 
the reader- s level of motivation 

interacting with 
the readability of the material. 

This writer (20) has proposed that readability is a 
"moment" at which time the reader^ S emotional, cognitive, 
and linguistic backgrounds Interact with each other, with 
the topic, and with the proposed purposes for doing the 
reaSingi and with the author's choice of seinantic and syn- 
tactic structures all within a particular setting. At such 
a "moment," the material is a constant on which two main ^' 
sets of forces are being exerted i one, the characteristics 
of the reader^ the other, the elements of the situation-^ 
actual and perceiv4?d. 

Although there seems to be a fairly high degree of a- 
greement as to what factors may influence readability , there 
is not the same degree of agreement v/hen it comes to putting 
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those factors into an operaticnal form* Educators v/ho at^ 
tempt to predict a message -s difficultir seek to use those 
characteristics of the written messaie that will place t>ie 
material along a continuum of selections v/hose "readability" 
scores hsva already been established (^) * Those who attempt 
"to measure readability seek to gain an estimate of the read-^ 
or*s understanding of that material as a function of^'^the 
reader's langi-iage competence, the subject master of the mes-- 
sages and the syntactic and morphological complexity of the 
message. These two approaches will be examined in an attempt 
to identify a means for judging the suitability of instruc-- 
tional materials for effectiy.e and efficient learning by a 
particular pupil population', 

PREDICTING THE DIFFICULTY OF TEXTS _ 

The most common means fo^ predicting the readability of 
materials is through the use of "standard" readability for- . 
mulas (IXf X3.f Ml* 21) which use factors such as vocabu- 

lary and sentence difficulty to sample "those characteristics 
of. reading material which make for ease or difficulty in read- 
ing Qomprehension (^) These two variables .are used because 
they seem to represent the two factors with the highest load- 
ings on regression equations (ISf Z^) ^ Vocabulary difficulty, 
however, is not estimated the same in all of the formulas, 
In some (1^, l^t ZZ) * difficulty means unfamlliarity , that 
is, a word is judged difficult if it does not appear on a 
list of frequently used words. In others ^) , difficulty 
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is estimated by word length, that is, a v/ord^is judged more 
difficult as the number of its syllables increases. The 
rationale given for these decisions is igj) i (a) that the 
length of a word seems to be a reflection of Its familiar- 
ity, (b) that as words are used more frequently they seem to 
become shorter, (c) that humans seem to repeat famil- 

iar words more frequently than unfamiliar ones, and (d) that 
sentence length seems to correlate very highly with sentence 
comp]exity. Therefore, "as long as predictions are all that 
is needed, the evidence that simple word and sentence counts 
can provide satisfactory predictors for most purposes is 
quite conclusive ( 23 ) ■ " 

In or^er to estimate the instructional level of the 
materials, ratios between the v/ord and sentence difficulties 
are compared to a population's performance on some criterion* 
The two most popular means for establishing grade equivalents 
has been to correlate the word/sentence difficulty ratio v/ith 
a population's performance on a graded series of comprehen- 
sion tests I or to compare the ratio to that found in a graded 
series of reading material . 

The use of standard formulas has been criticised on a 
number of accounts , One issue is the difficulty researchers 
seem to have' in defining the criteria of comprehensibillty 
(Ji ^» if M2L^9 M) • devising many of the f ormul'as , a 

percentage of correct answern on comprehension tests/has been 
used. Any comprehension test based upon a set of questions 
may be only as accurate as the questions themselvos since 

6 



^ Hittleman 

an unpredicta°bl-^ variation may toe introduced due to the 
uncontrolled ways items can be selected or phrased. Criterion 
scores on different tests made from the bame material may 
represent different amounts of knowledge, that is, another 
examiner might possibly devise a different set of questions 
from the same par^aages that would produce different compre'' 
hension results* AlsOi when formulas are standardized with 
one type of pupil populationi such as elementary school 
pupils I and then grade equivalents are extrapolated for use 
at other levels s the formula's validity for use with materials 
a^^ those other levels might be questioned , 

Another issue concerns the two common variables of ^the 
standard formulas, word and sentence difficulty,^ They are not 
seen as variables that can be ooiisidered to stand in a causal 
relationship to reading difficulty (2) . Since those two 
variables cannot be manipulated, as can other variables such 
as sentence structure, anaphora, and syntactic complexity, 
their role in causing reading difficulties is highly question- 
able. Also, in regard to the use of words to estimate dif- 
ficulty, the idea that shorter words are used more frequently 
and therefore are more predictable (with predictable implying 
familiarity) , may be a fallacy (^1^) . Long words are often a 
combination of two short words or a short word with affixes* 
Their "predictability" Increases in relation to their redun- 
dancy so it seems to be the redundancy of words that makes 
them "familiar", not their length. Also, there is evidence 
that sentences with morphologically derived word forms are 
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more difficiilt to process than those of non-derived words (21 ) 
Difficulty seems to be due not to word length but to word 
form* 

Using a formula to estimate the grade level of any given 
written material is a problem in that some formula users wish^ 
them to accurately predict a precise level of the reading 
matter. ^ Scores from a fortnula are uS&-bie.._only within a range 
of scores since their standai'd errorsof measurement ^afB^'o£:tan 
as great as one whole year (^) * 

Finally, some research has shown that formulas may be 
invalid for accurate appraisal of material with technical 
vocabulary (9) ? and they may not be sensitive to factors with- 
in subject matter areas (iJ) * vrhat this means is that while 
some formulas may rank materials of different content in an 
order of difficulty, they cannot distinguish, between and a- 
mong materials of the same content to provide a useful estimate 
of their probable difficulty levels. 

In summary, standard formulas seem to have major short- 
comings in that (a) they may be of dubious value when used 
with pupils or with materials that are dissimilar to those 
used in computing the formulas originally, and (b) they do 
not conolder difficulty caused by factors such as concept load, 
foritiat of the material, organization of the ideas , or the 
writing patterns. V/hile standard formulas may have some llm^ 
Ited appeal as predictors of readability, their usefulness 
does not allov/ one to consider why one pieces of instructional 
material is more or less readable than another, 
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OTHER P^DICTO'RS OF TEXTUAL DIFFICULTY 

Recently, much attention has been given to the role that 
syntactic complexity (factors other than sentence length) 
plays in determining the. readability of written materials. 
As^a result, there exists a formula of syntactic complexity. 
(6) and a means for establishing a syntactic density score (22) , 
These formulas, lik^ standard formulas, are intended to rank 
selections in an order of difficulty. 

The rationale for such syntactically based formulas seems 
to be that (l^) > (a) sentence length offers. little indication 
of the grammatical make-up and complexity of a sentence, and 
(b) a sentence's complexity cannot be established from a word 
count. The originators of the formulas based their selection 
of variables on the results of research dealing the difficulty 
of different sentence transf crmations . The formula of syn» 
tactic complexity considers counts of such elements as sim- 
ple sentences, simple transformations, coordinate clauses, nan- 
sentence expressions, prepositional phrases, noun modifiers, 
adverbial modifiers, modals, infinitives, gerunds, coordinate 
clauses, deletion in clauses, dependent clauses, participles, 
and clauses used as subjects. The sentence density estimate 
considers counts of such elements as .terminal units (T-units) , 
subordinate and main clauses, modals, be and have forms, 
prepositional phrases, possessive nouns and pronouns, advero- 
ials of time, and gerunds, participles and absolute phi^ases. 

A^'s an aside, it is interesting to note that the author 
of one standard readability formula has hypothesized v/hat 
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might be considered as a recognition of the importance of 
analyzing sentence complexity beyond that of length alone (Ij) . 
What is suggested is a "kernel distance theory" In which the 
relative positions of the subject, predicate verb, and object 
are considered in the recognition that syntax can be altered 
and .his will not show up in a count of sentence length. Sen- 
tences are deemed more difficult as the distance (counted in 

r 

number of separating words) increases between the subject and 
the verb, between the verb and the object , and between the 
beginning of the sentence and the subject. As of noWj this 
is an interesting hypothesis v/hich needs empirical verification. 

Based upon! the evidence that certain grammatical constraints 
seem to provide more difficulty than others for pupils to 
process (2* iji 12) * ^he syntactic complexity measures attempt 
to give various weights to different elements of syntax. This 
allows for judging the difficulty of sentences containing simi- 
lar elements in different positions within the sentences. How- 
ever, some of the ci^lticisms of standard formulas can be raised 
about syntactic complexity or sentence density measures. 
First I they do not measure readability under natural condi- 
tions, that is, the interaction between the reader and the writ- 
ten message is not sampled. Second, these measures do not 
take into consideration :the context in which the sentences oc- 
cur. Each sentence is analysed without regard for the type 
of paragraph structure, organizationi or function in which 
the sentences are grouped. Third , the sentence measures do 

i 

not account for the factor of concept load. Although the 
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means used in standard formulas to estimate word difficulty 
is inadequate, forrnulas that rely solely on syntactical ele-^ 
ments disregard this issue entirely* Fourth, the formulas 
at present are only heuristic. Little empirical evidence 
has been provided that indicates the assigned weights truly 
represent a real order of difficulty* 

Other researchers have atterrpted to predict readability 
through the study of the affects of lexical density i the 
role of different types of grammatical units, and the dif- 
ficulty of different transformations (£) * For example, in 
one study (29) * it was found that the diversity, of lexical 
units (the number of different words used in a passage) and 
some estimates of the naturalness of language (measures of 
the ratios between the number of verbs and adjectives, and 
between the number of r.ouns and verbs and the number of ad- 
jectives) provided valid estimates of the difficulty of con- 
tent area materials. It seems that lexical diversity re- 
flects the .density of .ideas, details, cr concepts within a 
passage. The greater the diversity, the greater the chance 
that the reader will lose information* In another study (1^ 
it was shown that a combination of measures of lexical den- 
sity together with measures of inter- and intra-sentence com- 
plexity Were able to distinguish and predict comprehension 
scores across different subject matter areas. It seems that 
the. combinations of factors which can predict readability of 
materials from different disciplines are not the same as 
those which can be used to predict differences in difficulty 
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within each discipliiie, Apparently, each discipline has its 
own set of factors for predicting reading difficulty, V/hat 
may be more important / th"e vai^ious combinations of linguistic 
factors seemed to accurately predict differences in the dif- 
ficulty of materials judged to be of equal difficulty by one 
popular standard formula. 

There have been various studies to investigate the rela- 
tive difficulty of different word classes as predictors of 
reading difficulty. Representative of these studies are two 
( 25f 3^) which reveal that young" pupils seem to have more dif 
ficulty processing some conjunctions than others, and that 
:'oung school pupils do not seem to^ integrate personal pro^ 
nouns in the same way as adults. 

In attempts to understand the effect of paragraph struc- 
ture and organization on the readability of material, some 
researchers have studied the use of concept classification 
schemes and semantic models of prose for predicting the 
readability of materials ^) ^ And, in order to use some 

aspect of the r*3 ad er- mate rial interaction, at least one re- , 
searcher has tried to develop a format for having the dif- 
ficulty of passages predicted through the matching of tar- 
get selections to a scale of passages with a predetermined 
order of difficulty (8) . It seems that trained judges are 
able to match the target passages to a level of difficulty 
by subjectively considering the sentence patterns, the type 
and density of concepts, and the author's style in presenting 
ideas, ^ ^ , , 



■ . Hittleman -12 

While these studies provide insights into factors that v 
might be 'affecting the readability of passagesi the insights 
have still been put to use only in attempts to predict reada- ' 
billtyi Predictions are only reliable when the characteris- 
tics, of the reading material and reader population used in 
establishing the regression equation are similar to those 
of the material and pupils with whom one is working, 

MEASURING THE DIFFICULTY OF TEXTS 

A great deal of research has attempted to validate the 
cloze procedure (the systematic deletion of every nth word) 
^^ sls a means for estimating^ the readability of material (2, 2f' 2)' 
The rationale for its use Is ^hat the ^close probedure takes 
into account the interaction between and among the reader, 
the material, and the reading situation. The estimates of 
readability obtained through its use seem to be much more re- 
liable than those . obtained through the use of standard, formu- -. 
las* It is looked upon not as a predictor of readability 
but as an accurate measure^ of readability (\22) » 

Since cloze procedure scores are percentages, some way 
had to be created for translating .them into meaningful scores 
of readability. Recent research C^) has identified scores 
that represent a desirable levelj of performance on instructional 
materials that account^ for variables such as: (a) the read- 
er's learning, retention, and transfer of information, (b) 



the reader's rate of: reading and 



response , (c)' the_reader • s 



preference for the subject matter, style, and difficulty ^of-----,^^ 
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the passage and willingness to study it, and (d) the effpcts| 
on the reader' # self concept and attitudes from having studied 
the material. Therefore i readability scoreg were shown to 
vary depending upon the grade level of the readers and the 
purpose for which the material was to "be used (textbook ready- 
ing, referenee reading, or voluntary reading) . I 

A variation of the cloze procedure has been developed 
thsLt^does not compare the reader's responses to the original 
material. Rather, the reader's responses are compared against 
all responses placed in the blanks by a criterion group ( 26) ■ 
This procedure, called clozentropy , deems a word correct to 
the degree that members -of^^the criterion group agrc^e it is' 
correct. The comprehensibility of the message Is considered 
relative tp a specific audience. This procedure seems to 
have great potential for Intercultural communicators in that 
messages can be matched to a;^ particular audience more ef- 
fectively than ever before. 

Although the cloze procedure measures readability, where- 
as formulas predict it, there are some cautions that must be 
considered when using it (^1 ) • First, cloze tests may con- ' 
tain many Items for ^hich there are no context clues* These 
items, then, may not /be usable as discriminating items. Second 
many deleted items ire not reading related. The completion 
of those items riflectsthe reader * s general background in- 
fomation and general language ability* Third, eventhough 
.the clozfe procedure has empirical. validity , it does lack face 
validity i Fourth, cloze Itmes may depend too much on short 
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range language and memory constraints. Fifth .the "every 
5th word," deletion procedure may not be suitabla for every« 
one i Finally, completion of the cloze items are greatly in- 
fluenced by the type and number of deletions. 

In part response to the last two critlcismi, one group of 
researchers (^) is attempting to determine . the deletion rate 
appropriate for different content areas. Preliminary evidence 
seems to show that science materials may need a greater space 
between deletions than does social studies or English materials. 
By varying the deletion rate within the different^ areas , the" 
pupils' performances seem to be affected* 

DETERMINING THE PADABILITY 

OF INSTRUCTIONAL MATERIALS . ' 

The aim of this paper has been to present ideas that 
^ have implications^f or the effective selection of Instructional 
materials. The evidence on readability research strongly 
suggests that the comprehension of a reader is affected by (!) i 
, (a) the content aref. being read, (b) the background of Infor- 
matlpn required of the reader, (c) the author's writing style, 
(d) the vocabulary specific to the content area being read, 
and (e)..the linguistic and morphological complexity with which 
the passages in the different content areas ^re ^generated , ^ 
.Evidence, also seems to show that the use of /counts of the 
number of words and syntactic elements may be ignoring the 
effect upon comprehension of where those elements occur with- 
in the sentences ( 2) • Therefore, in order to capture the ■ _ 
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"moment of readability" at which time a measure is.- taken 
of the interaction between characteristics of the reader 
and the written message under the Influence of a particular 
instructional situation , the following recoinmendations are . , 
made i * . . 

1. Avoid the use of predictive formulas which usually 
have arbitrarily assigned grade equivalents or' are capable of 
only establishing a rank order of difficulty according to 
some criteria. Although formulas will provide some indica- 
tion of the relative difficulty of different materials in 
relation to each other,, they will not provide useful infer- 
I matibn about whether or not those materials are readable by 
a group of pupils i ' = ; 

E^i Use some form of the cloze procedure , The cloze 
procedure is the only available procedure which can take in- 
to account, in a natural setting, the constraints -of the lan- 
guage system of the reading mattery the reading ability and 

i 

other characteristics of the reader, and the background in- 
formation needed by the reader. Whils^it. is usually con- 
sidered expediant\to count only "correct" responses ( correct 
in relation to whether or not the author's s^'eclfic language 
- is accurately reconstructed) j a count, of itfe^ihs that are con- 
sistent in frequency and meaning with those /of other members 
of the particular reading population can pr^Dvide teachers 
with valuable Information as to how the pupils can or cannot 
determine the author- i message because of^fact.ors within ' 
their common linguistic or informational sxperiences. This 
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use of the cloaentro-p y technique will allow teachers to 
adjust their measures of the various factors of readability 
to local Instructional conditions, 

3- Do not use the same criteria of success for all 
age groups, for all materials, and for all purposes* There 
are some established criteria available for use at different 
grade levels when reading for different purposes (^i ^) • These 
can provide teachers with quick Interpretations of cloze 
scores, provided the cloze procedure is used under the con- 
ditions recommended by that researcher. However , since evi- 
dence of other researchers indicates that close scores are 
also Influenced by the number and type of deletions and by 
the content area, educators may find, it useful to establish 
local criterion levels suitable for their instructionai pur- 
poses and materials. 

CONCLUStON ■ ' 

Instructionally , our purposes must always be to think of 
readability as ever/ changing. Our standards of what is 
readable and our judgments as to what Is understandable should 
always be relative to a particular instructional situation. 
With our current knowledge of the factors that are Interacting 
durlrig an act of reading, it is inexcusable to rely, solely 
upon some artiJicial and arbitrary means for olasslfying read- 
ing materials • We must always remember that, when pupils are 
engaged In a reading situation, they bring not only know- 
ledges and understandings of the content area and topic, > being , 
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read, but alio their anticipations and expectations of what 
are. the purposss and objectives of the reading lesson. We 
shou].d never eliminate the reader and the act of reading from 
our concept of readability. 
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